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ABSTRACT 

This review brings diverse research to bear on the 
contention that current achievement tests may underestimate students* 
subject-matter knowledge and problem- solving ability because of the 
mismatch between the symbolic form that typical achievement tests use 
and the specificity of students* symbolic encoding that arises from 
instruction and individual differences. More specifically, the 
possibility that alternative representations of science problems 
affect achievement estimates is examined for students varying in 
socioeconomic and ethnic/racial backgrounds. Theory and research are 
examined for the effects of symbolic encoding on information 
processing and the effects of translation among symbol systems on 
problem solving. Testing with alternative symbolic representations is 
placed in the context of the literature on minority group testing. 
Research on minority group testing has rarely focused on the 
possibility of alternative symbolic forms of tests or test items. 
There has been little attempt to determine the strengths of 
particular cultural and language groups and to develop tests that 
capitalize on them. Instead r research on minority group testing has 
concentrated on: (1) the validity of group differences in test scores 
(test bias); (2) the testing of linguistic minorities; and (3) the 
development of culture-fair tests. A 70-item list of references is 
included. An appendix describes the development of a taxonomy used in 
science textbooks to present concepts of heat and temperature. 
(SLD) 
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ALTERNATIVE STRATEGIES FOR MEASURING HIGHER-ORDER SKILLS: 

THE ROLE OF SYMBOL SYSTEMS 

Richard J. Shavelson, Noreen M. Webb, and Penny Lehman 



Problem solving in many subject-matter domains often requires 
the problem solver to transform a problem from its original 
symbolic representation (e.g., words) into an alternative symbolic 
representation (e.g.. iconic, mathematical) in order to arrive at 
a solution (Clement, Lockhead, & Monk, 1980; Hooper, 1981; Nesher. 
1982; Shavelson. 1981; Shavelson & Salomon, 1985). Consider, for 
example, the following word problem: "Start with one beaker of red 
solution and one beaker of water. Place a teaspoon of red solution 
from the first beaker into the second beaker. Then place a 
teaspoon of liquid from the second beaker into the first beaker. 
Is the amount of red solution in the lirst beaker equal to the 
amount of water in the second beaker?" The verbal presentation 
usually leads to a logical mistake. Recognizing that the word 
problem can be transformed into an algebraic representation leads 
readily to the correct solution. 



THE PROBLEM 

In education, words (verbal symbols) dominate instruction. 
If the symbolic representations used In instruction set boundaries 
within which students learn and remember concepts, this encoding 
specificity may place restrictions on students' abilities to 
translate the problem as given into a symbolic representation that 
admits to a solution. We believe that encoding specificity 
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characterizes students' knowledge especially when concepts are 
learned initially. Puller understanding coMes with repeated 
exposure to the material in different contexts and with repeated 
application of the concepts to different types of problems. With 
full understanding of the material, multiple symbolic 
representations of the same concept can be recognized and used to 
solve problems. 

Partial understanding characterizes students' knowledge 
acquired from many courses.^ We suspect that this partial 
understanding is dominated by the verbal and limited range of 
other symbolic representations used in classroom instruction. 
Moreover, we suspect that this understanding also depends on 
differences in how students symbolically represent informition to 
themselves (aptitude). As a consequence, students' understanding 
of a subject will depend greatly on the symbolic representations 
used in instruction, their preferred symbolic mode of representing 
that subject, and the fit between instruction and aptitude. 

If this characterization of students' understanding from 
instruction is roughly accurate, it has important implications for 
achievement measurement. Typical achievement-test items, in and 
of themselves, may require students to translate from the items' 
unique, predominantly verbal code to a more general (verbal or 
nonverbal) representation of the problem. Once the multiple- 
choice code has been broken, the student may also need to 
translate from this initial representation to some other symbolic 
form in order to solve the problem. If the purpose of achiev';*ment 
testing is to measure subject-matter knowledge, not verbal 
ability, typical objective tests may not this goal as well as they 
might. Rather, they may place irrelevant demands on certain test 



^Instruction, especially with limitations in depth of 
coverage and time allocated to important concepts, is often 
Incomplete. In addition, students' may hold misconceptions about 
the concepts to be studied that interfere with their 
understanding. Finally, their "mental effort** in acquiring the 
new concepts may be less than optimal. 
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takers, and thereby underestimate some students' subject-matter 
knowledge . 

The purpose of this review is to bring diverse research to 
bear on our contention that current achievement tests may 
underestimate students' subject-matter knowledge and problem 
solving ability because of the mismatch between the symbolic form 
typical achievement tests use and the specificity of students' 
symbolic encoding that arises from instruction and individual 
differences. More specifically, we investigate the possibility 
that alternative representations of science problems affects 
achievement estimates for students varying in socioeconomic and 
ethnic/racial backgrounds. Science was chosen because of its 
importance in educational reform, and because alternative symbolic 
representations play a central role in "doing science." 

REVIEW OP LITERATURE 

To address the question of whether different symbolic 
representations produce different estimate of students' 
achievement, first need to define what we mean by symbols and 
how different symbol systems might be identified and classified. 
This task is taken up first in the review. Next, we examine 
theory and research on (1) the effects of symbolic encoding on 
information processing and (2) the effects of translation among 
symbol systems on problem solving. He then place the issue of 
testing with alternative symbolic representations in the context 
of the literature on minority group testing. 

Symbols and Symbol Systems 

Definition . Gardner's distinction between symbols, symbol 
systems, and symbolic products helps clarify these concepts. A 
symbol is "any entity (material or abstract) that can denote or 
refer to any other entity. On this definition, words, pictures, 
diagrams, numbers, and a host of other entities are readily 
considered symbols" (Gardner, 1983. p. 301). 
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A ayabol system Is a set of symbols and rules for combining 
them. An additional required feature, according to Salomon 
(1979). is correspondence to a field of reference. The field of 
reference is important for giving meaning to the set of symbols 
and rules (see also Gross. 1974). As an example. Salomon describes 
the ambiguity of the meaning of red lights unless the field of 
reference' is known: traffic lights, boats at sea, windows of alley 
apartments . 

Finally, symbolic products consist of "all manner of symbolic 
entities that individuals create in order to convey a set of 
meanings, and that other Individuals imbued in the culture are 
able to understand. Interpret, appreciate, criticize, or 
transform" (Gardner, p. 301). Achievement tests, then, are 
symbolic products. This review addresses the adequacy of this 
particular symbolic product for its intended purpose, viz. 
eatlnating students' knowledge, skills, and problem solving 
abilities in a subject-natter domain. 

Symbol System Classifications . Classifications of symbol 
systems vary from broad classes, such as Gross' (1974) five 
primary modes of symbolic behavior (linguistic, social-gestural, 
Iconic, logico-mathemat ical , and musical), to more specific symbol 
systems, such as speech, print, drawings, diagrams, models, 
graphs, maps, numbers, geometry (Olson & Bruner. 1974) and 
pictures (Jay, 1973). 

Only rarely have typologies of symbol systems been tied to 
Instructional material. One of very few examples is Hooper's 
(1981) typology for mathematics: (1) equat ions--mathematlcal 
formalizations that represent precise relations between things; 
(2) graphs--simultaneou8 displays both of the general relationship 
between variables and the specific relationships between 
particular values of a set of variables; (3) tables--an 
Intermediate representation between equations and graphs that 
provides Information about the relations among values of a set of 
variables; (4) pictures--displays of specific instances, making 
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the problem concrete, and (5) diagram8--vi sual displays of the 
general characteristics of a problem that demonstrate abstract 
concepts such as Venn diagrams, pie charts, flow charts. (We have 
applied Hooper's taxonomy to representations of science concepts, 
see Appendix . ) 

A second example is Lesh, Post, and Behr's (in press) 
taxonomy of representations systems found in mathematics. Losh et 
al. distinguished five types of symbolic representations: (i) 
real scripts which organize knowledge around "real world events 
that serve as models for interpreting and solving other kinds of 
problem situations" (p. 6); (2) manipulative models (e.g., 
Culsenalre rods) which contain elements that correspond to 
relationships and operations in many everyday situations; (3) 
pictures which, 'like manipulative models, can be internalized as 
•images'"; (4) spoken languages including special languages such 
as logic; and (5) written symbols, which in addition to written 
English, may ii\volve specialized sentences such as x+3-6. 

More frequently in educational research and curricula, symbol 
systems are cast in terms of media. Fitzgerald and Vance (1970), 
for example, give an extensive list of media that may be helpful 
for teaching mathematics: television, computers, programmed 
textbooks, computer assisted Instruction, overhead projectors, 
audio tapes, film loops, film, radio, academic games, and 
Manipulative devices (e.g.. geoboards). 

These media are not single symbol systems, however, but are 
combinations of then. Film, for example, uses many symbol 
systems, including photography, gesture, speech, dance, music, and 
film-specific symbol systems (Salomon, 1979). 

Even the specific symbol systems mentioned above may not be 
single or pure symbol systems. Diagicams, for example, may not be 
a single symbol system, but may be a cluster of distinct symbol 
systems. Schematic drawings of electric wiring, pie charts, flow 
charts, and Venn diagrams can all be considered diagrams but they 
use somewhat different symbols (circles, squares, numbers, arrows) 
and even the symbols that they have in common may have different 
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neanings (compare a circle in a flow chart with a circle In a pie 
chart). As is discussed next, the literature on sywbol systems 
gives little guidance on deciding how to distinguish one symbol 
system from another. 

Dimensions of Sym bol Systems . To define a taxonomy of symbol 
systems that is relevant to a particular knowledge domain, it is 
necessary to clarify the critical dimensions that distinguish 
symbol systems from each other. The dimensions that have been 
proposed to differentiate symbol systems include resemblance 
(Goodman, 1968; Eisner, 1970; Sowell. 1974) and notatlonality 
(Salomon, 1979). 

By resemblance, we mean the extent to which symbols resemble 
their referents. At one extreme are symbol systems that resemble 
or Imitate their referents (Iconic), for example, replica models. 
At the other extreme are symbol systems that do not resemble their 
referents (analogue), for example, verbal and numerical systems. 
Salomon (1979) speaks of description vs. depiction, respectively 
(see also Chapanls, 1961). Sowell (1974) classifies symbol systems 
into categories of concrete, pictorial, and abstract, and Eisner 
(1970) classifies symbols as representational (realistic 
depletions), conventional where symbols stand for ideas or events 
In a particular culture (e.g., Valentine's heart), connotative 
where symbols result from distortions of conventional symbols 
(e.g., Picasso's animals), and qualitative where symbols represent 
some Idea or feeling. 

Goodman (1968) and Salomon (1979) argue that resemblance is 
an unsatisfactory way of defining symbol systems because 
resemblance Is ambiguous; symbols can resemble their referents in 
many ways. They argue instead that what really distinguishes 
between depiction and description is notatlonality. Symbol systems 
can be notational or nonnotat lonal or somewhere between these two 
extremes (Salomon, 1979). In notational systems, the symbols are 
discrete and discontinuous and there is a one-to-one 
correspondence between symbols and their referents. For example, 
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each note on a musical score corresponds to a nuslcal pitch. In 
nonnotational systens, symbols are not disjoint but are 
continuous, and each element does not correspond to one and only 
one referent (Gardner, Howard, & Perkins, 1974). For example, 
pictures are nonnotational because each element could represent 
many things (a line In a painting can represent a contour or fold 
OP depth), and the picture could lead to multiple meanings 
(Salomon, 1979, p; 33). As an intermediate case. Salomon argues 
that language is only partly notational because the elements are 
often ambiguous. 

These dimensions are of little help in deciding how to 
distinguish between symbol systems. Salomon (1979, p. 35) admits 
that notationality is only one of many possible ways to 
distinguish among symbol systems. Furthermore, the concept of 
notationality is too abstract to help define taxonomies of symbol 
systems for particular knowledge domains. Clearly, then, while 
the literature presents examples of symbol systems, no one has 
clarified the critical dimensions that distinguish them, and we do 
not Intend to attempt to do so either. Rather, this review leads 
ua to the conclusion that we need to develop a symbol system 
taxonomy that is heuristic for our specific purposes. 

Encoding Specificitv 

Proposition vs. Multi-Representation Controversy . An age 
old controversy continues about how information is encoded 
internally. One view is the propositionalist position: that all 
information is encoded in a single underlying form of mental 
representation called propositions (Anderson & Bower, 1973; Chase 
4 Clark, 1972; Pylyshyn, 1973, 1976: Reed, 1974; Palmer, 1975; 
Norman & Rumelhart, 1975; Olson & Bialystok, 1983). Propositions 
are abstract and do not correspond directly to words or pictures 
or any other symbolic mode. Pictorial information, for example, 
is "alleged to be processed sequentially and complexly and In a 
way that is modality-independent. Each stimulus needs to be 
decomposed into higher-level then lower-level propositions until 
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reduced to primitive propo itions about points, angles, and 
nuneroua other dinenslons" (Salomon. 1979, p. 67). 

The coapetlng view of encoding posits that information is 
encoded in terms of properties that are modality specific. Most of 
the proponents of this view are concerned with the processing of 
pictorial information and maintain that images are encoded in 
different mental represeiitat ions from, say, verbal information 
(Shepard, 1978; Brooks, 1967; Kosslyn. 1975; Kosslyn & Pomerantz, 
1977; Palvlo, 1971, 1976; Bower, 1972). 

Although the proposltionallsts and the 
■ultl-representatlonallsts disagree about the deepest level of 
■ental representation (proposition vs. modality-specific 
representation), both sides agree that, at a higher level, there 
exist different kinds of mental representations or mental models. 
No one seems to deny that there is a phenomenon called mental 
Imagferv (Anderson, 1978, p. 251, emphasis in original); the 
controversy centers on the deepest level of the Internal 
representation. Even proposltionallsts agree that a learner may 
have distinct mental representations of pictures and words at some 
level above the propositions. An extreme case is Johnson-Laird 
(1983) who, even though a propositionalist , hypothesizes the 
existence of six types of physical mental models (corresponding to 
the physical world) and four types of conceptual mental models 
(corresponding to abstract concepts). 

We are concerned with the level of encoding that admits 
■ultlple mental representations, not the most basic level of 
encoding, so the controversy is not relevant to our project. We 
base our project only on the assumption that, at some level, 
Information that is presented in different symbolic modes is 
encoded in different mental representations ("encoding 
specif icity" ) . On that point cognitive psychologists are in 
agreement . 

Tulving's Encoding Specificitv Principle . The notion of 
symbolic encoding specificity should not be confused with 
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Tulvlng'a Encoding Specificity Principle. In their encoding 
specificity principle. Tulving and Thomas (1973, p. 353) 
hypothesized that Inforaation Is encoded according to the context 
and requirements of the task: "What Is stored Is determined by 
what Is perceived and how It is encoded, and what is stored 
determines whac retrieval cues are effective in providing access 
to what Is stored." They Illustrated their principle with 
homographs: words that have oAfferent meanings: for example, if 
the word "violet" is encoded as a color name, it normally will not 
be retrieved as a type of flower or girl's name. Although there 
is a common element between Tulving's principle and our hypothesis 
about symbolic encoding 5»peclf Icl ty , that features of the learning 
task influence encoding, Tulving was not concerned with the 
symbolic representation of the learning material. 

Evidence for Symbolic Encoding Soeclflcitv . Wc cannot find 
research directly related to symbolic encoding specificity. The 
closest evidence is indirect and comes from studies finding that 
people acquire different knowledge from different media. The 
results of Thorndyke and Hayes-Roth's (1982) map-learning study, 
for example, support their hypothesis that when learning from a 
map. people encode global spatial relations, Images that can be 
scanned and measured like a physical map: whereas when learning 
through navigation, people acquire knowledge about the routes 
connecting different locations. 

The Need for Research . The symbolic encoding specificity 
hypothesis as applied to multiple symbol sys tem8--that material 
presented in different symbolic forms will be encoded 
differently — has enormous implications for Instruction, but the 
link has not been made . Presenting information to students in 
multiple symbol systems, say. pictures and numbers, requires not 
only understanding of the material in each symbol system, but 
requires an extra step of translation to connect the two. The 
studies comparing the effectiveness of combinations of symbol 
systems (e.g., pictures and words) to single symbol systems (e.g., 
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words) do not take this step into account in hypotheses about how 
information is processed nor in their interpretation of results. 
As only a few researchers have recognized (Wiebe, 1983; Soweil, 
1974), this extra step Bay aake it more difficult, not easier, to 
learn from multiple representations. 

Translation among Symbol Systems 

Consider Charlie, a student who has just learned to determine 
the area of a rectangle and is then given a picture of a 
(rectangular or square) living room and asked to determine how 
large a rug would be needed *-o fit it wall to wall. The student 
is fully capable of determining the area of a rectangle (area =' 
length times width); and has first hand experience of rugs in 
living rooms. Yet he may not see that his geometric knowledge can 
be applied directly to the problem at hand. To make this link 
requires that he trans late between one symbol system (geometry) 
and others (verbal and Iconic). And yet, Charlie may be symbol 
system bound. If he is not given two of the triple — area, length. 
width--in a computational problem or a schematic of a rectangle, 
he nay not link his geomy^tric knowledge to the problem at hand. 
Once the translation is made, however, the problem becomes 
trivial . 

Amy, a fifth grade student in the high math group, was 
confronted with a problem that went something like this: Given 
two workers moving furniture Into a house, determine the minimum 
tine needed to complete the Job when each task required a 
specified amount of time, some tasks required the two movers to 
work together, and some tasks had to be done before others. The 
problem was solved when it was recognized that a piece of paper 
could be used to stand for each task, and that the length of the 
paper could be used to denote the time required to complete the 
task. By placing the pieces of paper above a time line and moving 
them around, the minimum time required to complete the job (under 
the constraints given) could readily be determined. With this 
problem. Amy might have recognized, given time and motivation, 
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that some other symbolic representation, such as algebra, might 
also have yielded a solution to the problem. Nevertheless, by 
translating from the word problem to the spatio-temporal symbol 
system, the problem became tractable and the solution 
straightforward. That is, the rules for manipulating the 
spatio-temporal symbols, including moving the rectangles around to 
meet a set of criteria, provided the operations needed to move 
from the problem's givens and constraints to its solution. 

These are but two examples among many where the requisite 
knowledge and experience of the problem solver are more than 
adequate to solve a problem, but the symbolic form of the problem 
as given (here predominantly verbal) does not admit to a ready 
solution. The "real" problem for the problem-solver is to 
recognize that the same problem can be represented by another 
symbol system, one in which the problem can be solved. 

Tentative Definition of Symbol System Translation . These two 
exanples convey, informally, what we mean by translation from one 
symbol system to another. Heuristical ly , we define translation 
from one symbol system to another as the process of mapping the 
information contained in one or more symbolic forms (e.g., 
geometry) into one or more, other symbolic forma (e.g., 
verbal/pictorial). Two prerequisites for translation are 
relevant: (a) adequate domain knowledge (declarative and 
procedural), and (b) experience. 

He do not assume a one-to-one mapping of symbols in one 
system onto the symbols In another system. The mapping syntax, 
for example, includes transformations such as combining verbal and 
numeric symbols (task and time in Amy's problem) into a primarily 
spatio-temporal representation (pieces of paper of varying 
lengths ) . 

He also do not assume that only one napping from a given 
symbolic representation to another exists for problem solution. 
Individual differences will play a role as will the context of the 
problem solving situation. That is, individual differences in 
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**mental ly** representing information symbolically and manipulating 
it will Influence the translation process. Hence, students may 
arrive at a problem solution along different symbolic paths. 

Research on Translation , Lesh (in press) defined 
'^translation among representations" in mathematics as a problem- 
solving process of: (1) translating from the ^given situation' to 
a mathematical model, (2) transforming the model so that desired 
results are apparent, and (3) translating the model based result 
back to the original problem situation to see if it is helpful and 
makes sense** (p. 1). Lesh, Post, and Behr (in press, p. 8) 
identify five steps in the translation process^ corresponding to 
modeling a mathematical problem: (1) simplifying the problem by 
ignoring Irrelevant information, (2) mapping between the givens 
and the ^'model,** (3) transforming the properties of the model to 
arrive at a result, (4) translating the result back to the givens, 
and (5) evaluating the fit of the result to the givens. 

Lesh uses the following problem to exemplify this definition 
(Leah, in press, pp. 1; see also Lesh et al., in press): 

Al has an after-school Job. He earns $6 per hour if he 
works 15 hours per week. If he works more than 15 
hours, he gets paid **tlme and a half** for overtime. How 
many hours mwst Al work to earn $135 during one week? 

After paraphrasing the problem and mulling it over, the 
student night arrive at a mathematical statement such as: 

(6 • 15) -t- 9(x-15) - 135 
At this points the student solves for x by applying a series of 
algebraic trans f ormat ions to the statement: 

This arithmetic sentence can then be solved using arithmetic 
transformations: x « 20; Al would have to work 20 hours. Beyond 
the initial mulling over process, the solution, according to Lesh, 
Involves three significant translations: one from English 
sentences to an algebraic sentence, another from algebra to 
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arithmetic, and a final translation from arithmetic back to the 
original problem. 

Leah (in press, p. 2) provides another example in which the 
same problem can be translated into more than one symbolic 
representatlon--algebraic or geometr ic--that admits to problem 
solution. . 

A boat, traveling upstream on a river, takes two hours 
to reach its destination eight miles away. The return 
trip downstream takes one hour and twenty minutes. What 
is the speed of the river current? 

The student might solve the problem using two unknowns: x » 
speed of boat; y « speed of current. Two algebraic sentences can 
be used to model the problem: 

2(x,-y)-8 and 4/3(x+y)-8. 
The solution, x « 5 and y « 1. can be readily attained. 

Alternatively, the algebraic model can be translated into a 
geometric model by graphing each of the two equations. The values 
that satisfy both equations will be the coordinates of that point 
common to both graphs: 
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Finally, note that If the student continues solving 
algebraically for x and y in the two equatlona, there is a 
cor.^esponding graphical representation. For example. 

X - 4 = X 6 
2x a 10, 

produces one graph with a horizontal line at y » 10 (y is a 
constant), and a second graph in which Zx produces a line passing 
through 0,0 ... 5,10. Although this would he a tedious process 
with pencil and paper, a computer cz.n produce a graph 
corresponding to the algebraic solution steps instantaneously for 
the student. When this happens, Lesh (jn press, p. 2) notes that: 

[N]any students ... have noticed that, for any given 
problem, intersection points for the pairs of lines at 
each solution step always lie on a single vertical line, 
which turns out to be at the solution point for x. Some 
... think about and describe why this invariant feature 
occurs. So ... students ... generate significant new 
questions, and ... [use] informal language to describe 
rather deep principles related to: (1) Invariance under 
■appings among isomorphic systems, and (2) Invariance 
under transformations within a given system. 

In a study comparing two groups (n^ « ng = 10) of nirth 
graders sti'dying comparable sequences of instruction on 
slnultaneous linear equations, both groups received exactly the 
same examples, exercises, problems and applications. Th(. groups 
differed in the order the various items were given, the part of 
the activity that was done by computer (rather than by the 
student), and the extent to which the graphs of solution steps 
were plotted. The "utilities" group used the computer as a tool, 
performing computations and assisting in problew solving and 
translating between equations and graphs. The "computation" group 
was guided by the computer through the steps needed to compute 
solutions to pairs of linear equations. "In a sense, the students 
in the 'computation* group performed the exact role that ... [the 
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computer] had performed for ths 'utilities' group, and the 
computer performed the role that the 'utilities' group students 
had performed" (p. 7). The "utilities" group outperformed the 
"computation" group on applications problems. They also 
outperformed "their originally comparable peers on the computation 
half of the test.... Some of the most impressive observations 
(from interviews] that the students in the 'utilities' group made 
had to do with the kinds of 'invariance under translation? and 
transformations' facts that were described [above]. No similar 
observations were made by anyone in the 'computation' group" (p. 
7). 

Based on this and earlier work (Behr, Lesh. Post and 
Wachsmuth. 1985; Post, 1986), Lesh argues that "the ability to do 
these translations are significant factors influencing both 
mathematical learning and problem-solving performance" (Lesh et 
al.. in press, p. 7). Indeed, students able to solve mathematics 
problems do so by representing the problems not in a single symbol 
system, but in several systems, each corresponding to different 
parts of a word problem (Lesh, Landau, and Hamilton. 1984). 

Most students, however, not only have difficulty 
understanding word problems and pencil and paper computations , they 
lack an understanding about models and languages needed to 
represent and manipulate ideas in problems (Behr et aL. , 1985; 
Post. 1986). To diagnose these difficulties, Lesh et al. (in 
press, p. 8) recommend presenting an "idea in one representational 
mode and asking the student to render the same idea in another 
mode. Then, if the diagnostic questions indicate unusual 
difficulties with one of the [symbolic representations] ... other 
[representations] ... can be used to strengthen or bypass it. 
, Herein lies the kernel of our idea about achievement testing. 

Achievement tests present problems in one dominant symbolic form: 
verbal, multiple-choice (often word) problems. Not all students 
have experience with such representations, especially if other 
representations have been taught with greater frequency, or if 
this representation has not been taught specifically. Hence, 
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switching from the usual verbal multiple-choice word problem to 
other representations might reveal knowledge that otherwise viould 
be judged absent. 

In addition to research on mathematical translation there are 
two related psychological literatures that provide important 
background for our work on translation. The literature on 
transfer is one, and the literature on insight is the other. Both 
are voluminous; we do not intend to review them comprehensively. 
Rather, we take a representative study of each and show how it 
does and does not relate to our notion of translation. 

Transfer . Transfer may be defined as "the extent to which 
the learning of an instructional event contributes to or detracts 
from subsequent problem solving or the learning of subsequent 
instructional events" (Royer. 1979, p. 53). A wide variety of 
types of transfer have been identified, e.g., "vertical," 
"lateral," "specific," "nonspecific," "near," "far." Both lateral 
and far transfer come closest to what we mean by translation. 

Lateral transfer is "a kind of generalization that spreads 
over a broad set of situations at roughly the same level of 
complexity" (Gagne, 1965, p. 231). For example, lateral transfer 
occurs when a child realizes that her knowledge of fractions is 
relevant to dividing up a prized. Jointly owned, marble 
collection. Far transfer arises when an individual recognizes 
that the organization of the stimulus complex in memory maps onto 
a differently organized stimulus complex presented by a task. For 
example^ far transfer .occurs when the student realizes that 
two-column addition applies to the solution of a word problem, or 
more generally, when an individual realizes that school learning 
applies to real-world problem situations (as in the geometry/rug 
example above . ) 

The definitions of lateral and far transfer seem to fit 
situations we have described as requiring symbol system 
translation, but are considerably broader. The little research 
that has investigated these types of transfer has focused on 
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content or knowledge structures, however, and not on symbol 
systems . 

DlVesta and Peverly (1984), for example, manipulated the 
context in which novel information was learned to examine the 
effect of context on transfer. For instance, a concept was 
taueht in one context, that of antique dealers. Far transfer, 
then, was defined as recognizing instances of a concept in a new 
context (e.g., magic shows). DiVesta and Peverly (1984) 
hypothesized and found that ''learners will acquire limited 
knowledge of a concept if they practice (encode) on examples 
appropriate to only the single context specified in the rules.... 
[Practice] on examples from a variety of contexts (encoding 
variability) will result in decontextual Ization due to the 
availability of multiple retrieval paths" (p. 109; see also 
Baddeley, 1982; Kerr, 1982; and Smith, 1982). 

Suppose that, instead of varying the content of the contexts 
such as "antique dealers,** and '*maglc shows," we varied the 
symbolic representation of the examples during practice. In this 
case, we would have one group apply the concept to verbal (same) 
exemplars (without the context tag), another group apply the 
concept to iconic (different) exemplars, and a third practice on a 
mixture of the two. We would, then, have an experiment that 
tested some aspects of our notion of symbol system translation. 

Insight . Insight refers to **the process by which the 
meaning, significance, pattern, or use of an object or situation 
becomeiii clear; or the understanding thus gained. In Gestalt 
theory, insight was originally described as happening suddenly, 
and a3 a novel reaction not based on previous experience** (English 
and English, 1966, p. 264). Sternberg (1984, p. 277) claims that 
three processes underlie insight: selective encoding, selective 
combination, and selective; comparison. Selective encoding 
"involves recognizing those elements of a problem that are 
relevant for task solution, and those elements that are not** (p. 
277). Selective combination '*involves figuring out how to combine 
Information that has been selectively encoded" (p. 277). And, 

17 



21 



most germane to translation, selective comparison "involves 
figuring out how new information can be related to old 
Information" (p. 277). Sternberg's example of selective 
Gonparison reveals symbol system translation: 

Kekule*s discovery of the structure of the benzene ring 
hinged upon his recognition that a dream he had of a 
snake reaching back and biting its tail provided the 
basis for the geometric structure of the ring [p. 277]. 

Sternberg and Davidson (cited by Sternberg, 1984) 
demonstrated that the three insight processes were additive by 
manipulating the amount and kind of cuing subjects received before 
attempting to solve problems like: 

Water lilies double in area every 24 hours. At the 
beginning of the summer there is one water lily on a 
lake. It takes 60 days for the lake to be come covered 
with water lilies. On what day is the lake half 
covered? 

The problem can be solved by recognizing that the problem can 
be translated to some form of numerical representation. For 
example, if each day (24 hours) the area doubles, then from day 59 
to day 60. the lilies will double to fill the lake; on day 59 the 
lake iiust have been half full. 

Conclusion. The construct, symbol system translation, 
focuses attention on the roles of: (a) prior knowledge and 
experience, (b) the learning context that produces specific 
symbolic encoding of new knowledge, (c) the symbolic demands of 
the problem solving context, (d) the processes that map knowledge 
learned In one or more symbolic forms onto another symbolic 
representation that includes cognitive operations needed for 
problem solution, and (e) individual differences in 
problem-solvers' symbolic encoding aptitudes. The construct 
appears to offer promise both in focusing theory and research on a 
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process that has not been directly researched to date, and in 
explaining, at least in part, why individuals have difficulty 
solving novel problems. 

Symbolic Representation and Minority Group Testing 

We suspect that some students and some groups of students may 
better process Information, and hence be tested more accurately, 
In primarily nonverbal rather than verbal symbolic forms. 
Candidate groups of students who may be disadvantaged when tested 
In verbal form are minority students or students from non-majority 
cultural or language backgrounds. This supposition follows from 
Gardner's (1983) notion of symbolic products (such as achievement 
tests) as consisting of symbolic entitles comprehended by 
individuals in a particular culture . Yet, the research on 
minority group testing rarely toucnes on the possibility of 
alternative symbolic forms of tests or test items. Rather, the 
literature on minority group testing has focused on (1) the 
validity of group differences in test scores (test bias), (2) the 
testing of linguistic minorities, and (3) the development of 
culture-fair tests. We address each topic in turn. 

Test Bias . A huge literature has developed out of the 
recognition that group differences on intelligence and achievement 
test scores exist. Much of the research has tried to determine 
the accuracy (validity) of differences in test scores across 
ethnic/racial groups (Cole, 1981). When a test yields group 
differences that are Inaccurate, the teat Is considered biased. 

At least three types of bias have been identified. The 
concern here is whether teat predict eventual criterion 
performance equally well for different groups (American College 
• Testing Program, 1973; Campbell. Crooks, Mahoney, & Rock, 1973; 
Jensen, 1980; Kalllngal, 1971; Klrkpatrlck, Ewen, Bennet, & 
Katzell, 198; Linn, 1975; Linn & Werts, 1971; McNemer, 1975; 
Stanley, 1971; Stanley & Porter, 1967; Temp, 1971). If groups 
show differences In predictor test scores, the question is whether 
the predicted differences in criterion scores are accurate (Cole, 
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1981). A second type of bias concerns the internal test 
structure. If the relationships within a test differ across 
groups of examinees, the test might be considered biased 
(Humphreys & Taber, 1973; Jensen, 1980). 

The final type of bias concerns the item. Items that 
distinguish between groups may not mean the same thing for 
different' groups (Cole & Nitko, 1981; Rudner, Getson, & Knight, 
1980). Even when group differences in item responses have been 
found, however, they have rarely been interpretable . One 
exception is Scheuneman's (1979) findings that Items worded 
negatively and items with unfamiliar formats on a school ability 
test showed bias against young black students (see Cole, 1981). 
Although Scheuneman's results suggest that features of an item may 
disadvantage some groups, that study focused on variations of 
verbal items, not alternative symbolic forms. 

In summary, although a large number of statistical techniques 
have been developed to detect test bias and to determine the 
appropriate statistical corrections in test scores to ensure fair 
selection of individuals from different cultural groups, the work 
In test bias has rarely investigated the mechanisms causing 
differences between groups nor has it investigated how to design 
tests that maximize the performance of all groups. 

Testing of Lingu istic Minorities. One area of minority group 
testing does focus on the sources of difference between groups: in 
particular, the language background of the examinee. 
Psychologists have long known that standard English tests may not 
accurately reflect the abilities of achievement of students whose 
native language is not English or who are not fluent in English. 
Olmedo (1981, p. 1083) raises the question about whether any test 
"can be used to Infer constructs that are conceptually equivalent 
across diverse cultural and linguistic groups.- In 
operatlonallzlng the issues surrounding testing of linguistic 
minorities, Olmedo and others are concerneu mainly with the 
language of the test, the difficulty with developing equivalent 
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versions of tests of different languages, and the need to match 
the language and cultural background of the examiner and examinee. 

The issues in testing of linguistic minorities, then focus on 
the language of testing, rather than the symbolic form of the 
test. The concern is whether the words should be in English or 
Spanish or nonstandard Bnglish (e.g., the Black Intelligence Test 
of Cultural Homogeneity or BITCH Test), not whether the problem 
should be presentv-d primarily in words or primarily in some other 
symbolic form. 

Cross-cultural Testing. The only area of testing that 
addresses the symbolic form of the test is the development of 
culture-fair tests. The developers of cross-cultural tests have 
tried to eliminate factors that distinguish between cultures, 
including language, reading, speed, and test content (Anastasi, 
1982). Culture-fair tests, then attempt to minimize language or 
reading requirements, are not speeded, and try to avoid using 
information that may be specific to particular cultures. Typical 
culture-fair tests are nonverbal and are figural or pictorial 
(e.g., Leiter International Performance Scale. Culture Fair 
Intelligence Test, Raven's Progressive Matrices, the Goodenoough- 
Harris Drawing Test). As Anastasi and others have pointed out, 
however, even nonverbal tests may be heavily culturally loaded 
(Anastasi, 1961; Irvine, 1969; Jensen, 1968; Ortar. 1963; Vernon. 
1969). Pictures and other nonverbal symbols mean different things 
to people from different cultures (Miller. 1973; Segall, Campbell. 
& Herskovitz, 1966). 

The attempt to eliminate cultural factors from a test — to 
develop the lowest common denominator~-is to move away from being 
able to Identify what an examinee knows or can do. Because "each 
culture and subculture encourages and fosters certain abilities nd 
ways of behaving; and it discourages or suppresses others" 
(Anastasi, 1982, p. 344), it is important to develop tests that 
capitalize on the strengths of students from different cultural 
backgrounds so that an accurate measure of ability can be 
obtained. Culture-fair tests do not do this. 
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Conclusion . Although the research on minority group testing 

recognizes that standard verbal tests (usually in English) may 
disadvantage students from non-majority cultural and language 
backgrounds, there has been little attempt to determine the 
strengths of particular cultural and language groups nor to 
develop tests that capitalize on then. Most alternatives tests 
are verbal, just translated into another language. The few 
nonverbal tests that have been developed try to eliminate cultural 
advantages rather than taking advantage of thera. 
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APPENDIX 



DEVELOPMENT OF TAXONOMY USED IN SCIENCE TEXTBOOKS: 
CONCEPTS OF HEAT AND TEMPERATURE 

This study is part of a project aimed at improving the 
■easureaent of problem solving skills in achievement testing. We 
suspect that estimates of achievement from traditional multiple- 
choice tests may underestimate students' achievement because 
students, when short of full understanding, may be bound by the 
symbol system in which they learned the material, or by the way 
they remember the material. The purpose of this study was to 
develop a rough taxonomy of symbolic representations of physical 
science concepts used in high school textbooks. 

The Problem 

If achievement tests are intended to measure students' 
subject-matter understanding, not verbal ability, many of these 
teats may miss their goal. Typically achievement tests present 
problems in a unique, verbal form with a stem and multiple 
alternatives. Yet if the student learned (encoded) the concepts 
using a different representation, the student must "translate" to 
a new representation before being able to answer the question. 
What may be needed is an achievement test that offers multiple 
syabolic representations of the same content domain to ascertain 
the dependency of student achievement estimates in symbolic 
representations. A first step in this research is to characterize 
how physical science concepts are represented in instruction. 
This taxonomy of representations, then, might guide the 
construction of achievement test items in alternative symbolic 
forms. 

A review of literature on symbol systems (Shavelson, Webb, 
and Lehman, this report) showed that the dimensions of symbolic 
representations identified in research are too abstract to guide 
the development of a testing instrument. Only rarely have 
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typologies of symbol systems been tied to instructional material* 
Hooper (1981) is an exception. She developed a taxonomy of 
symbolic representations for mathematics which included Ihese 
categories: equations* graphs, tables, pictures, and diagrams. 
Hooper^s categories served as a starting point in this study for 
developing a taxonomy of symbol systems used to teach a set of 
science concepts* This category system vras applied to the 
concepts of heat and temperature in three textbooks widely used in 
the Los Angeles Unified School District (LAUSD). Although 
students receive information from a variety of sources (e.g., 
teacher » textbook, other students, reference materials, class 
demonstrations) we chose to examine popular textbooks because of 
their wide application and science teachers* dependency on them. 
Moreover, because of the difficulty in defining a general taxonomy 
relevant across all areas of science, we focused on heat and 
temperature. This topic is covered in nearly all science 
curricula at the secondary level, is not extremely difficult to 
learn, admits to multiple symbolic representations of its 
conceptSt and has been the focus of research on misconceptions 
held by students. 

Method 

Textbooks . Three textbooks, Physical Science (Scott, 
Poresman, 1986), General Science (Allyn & Bacon, 1985), and Focus 
of Physical Science (Merrill, 1986), were chosen for the study 
because they are used throughout the LAUSD in the high school 
physical science course, Modern Science. This course Is targeted 
at students of average to below average abilities who most likely 
will not attend college. 

Procedures . The first step in developing a taxonomy was to 
apply Hooper^s work to the symbolic representations of heat and 
temperature concepts used in the textbooks » and then to modify 
this taxonomyt as needed. The next step was to use the modified 
taxonomy, determine the frequency with which each type of symbolic 
representation was used for each key concept. 
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All examples of symbolic representations of key concepts in 
heat and temperature were extracted from each of the textbooks. 
(Verbal representations were not included in this exercise, but 
were understood to be a major symbolic representation of science 
concepts in textbooks.) Sixty-eight examples of alternative 
symbolic representations were found. Three raters then 
independently sorted the examples into categories of distinct 
symbolic representations. Although Hooper's labels were used as a 
starting point to name the categories, the raters were encouraged 
to modify the categories as needed. The ratGrs then compared 
their results, discussed discrepancies, and came to a consensus on 
categories of symbolic representation used for heat and 
temperature n the three textbooks. 

This taxonomy was then applied to the presentation of a set 
of specific concepts within heat and temperature to determine 
which symbol systems were used for each concept. The following 
eight topics were the focus of this investigation: (1) definitions 
of heat and temperature, (2) thermometers and temperature scales. 
(3) specific heat, (4) heat transfer, (5) pressure, (6) molecules 
and movement, (7) heating systems, and (8) insulation. A tally 
was made of the frequency with which each symbol system was used 
to represent each concept. 

Results 

A Taxonomy . six basic classifications of symbolic 
representations, similar to Hooper's were identified: verba\ 
diagrams, photographs, tables, graphs, and equations. However, 
the raters divided two of these classif lcations--dlagram3 , and 
, equations--lnto finer categories based on distinctive 
characteristics . 

Diagrams were defined along three dimensions: concrete vs. 
abstract; static vs. dynamic; and labeled vs. unlabeled. Although 
only four types of diagrams were found in the texts, all possible 
combinations of these dimensions produce eight distinct types of 
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diagrams (see Figure 1). Equations were further defined as being 
expressed In either words or symbols (see Figure 2). 



Insert 


Figure 1 




Insert 


Figure 2 



The only tables found in the three textbooks contained list*? 
of words and nuabers (see Figure 3). Nevertheless, the raters 
identified a second possible type of table, a contingency table. 

Insert Figure 3 



Ten distinct types of symbolic representations, then, were 
found in the textbooks: (1) Verbal; (2) Diagrams that are 
concrete, show movement and have labels (n-10); (3) Diagrams that 
are concrete, do not show movement, and have labels (n«9); (4) 
Diagrams that are concrete, show movement, and are unlabeled 
(n-4); (5) Diagrams that are abstract, show movement, and are 
unlabeled (n-2); (6) Photographs (n-33); (7) Tables (n-6); (8) 
Equations with words (n-l); (9) Equations with symbols (n-l); and 
(10) Graphs (n-2) . 

However, the full taxonomy Includes some categories for which 
examples could not be found. In all, the taxonomy contains 
fifteen distinct symbolic representations: 



(1) 


verbal 




(2) 


diagram - 


concrete/static/ labeled 


(3) 


diagram - 


• concrete/static/unlabeled 


(4) 


diagram - 


concrete/dynamic/labeled 


(5) 


diagram - 


■ concrete/dynamic/unlabeled 


(6) 


diagram - 


• abstract/static/labeled 


(7) 


diagram - 


■ abstract/static/unlabeled 


(8) 


diagram - 


■ abstract/dynamic/labeled: 


(9) 


diagram - 


■ abstract/dynamic/unlabeled 


(10) 


equation 


- words 
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(11) equation - symbols 

(12) table - contingency 

(13) table - list 

(14) graph 

(15) photograph. 

Frequency of Symbol ic Representations , Table 1 shows 
the symbol representations found in each text for ^ach key 
concept . 

Insert Table 1 



The verbal representations are counted in number of 
paragraphs while all other symbol systems are counts of 
Individual representations* Table 2 provides a summary of 
the frequency of each symbol system across all key concepts 
and textbooks* 



Insert Table 2 



Another way of characterizing the heavy verbal 
presentation in the texts is to consider that all eight key 
concepts were represented verbally. The next most frequently 
used symbolic representation was photographs, most of which 
were more cosmetic than substantive. Diagrams classified as 
concrete/static/labeled were used to represent three of the 
concepts . Diagrams --concrete/dynamic/ labeled and diagrams-- 
abstract/dynamlc/unlabeled appeared for two concepts each. 
The remaining four symbolic representat ions » diagrams- 
-concrete/dynamic/unlabeled, tables in list form, equations 
with words, and equations with symbols, each occurred for one 
key concept . 
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Conclusions 

Although taxonomies of symbol systems used to represent 
science concepts were not found 1n the science-education 
research literature, this study demonstrates that it is 
possible to define such a taxonomy. It is a lengthy process 
and most likely needs to be tied directly to specific 
concepts of interest. It Is obviously difficult to quantify 
our findings. However it is clear that the textbooks relied 
almost entirely on verbal exposition. 
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FIGURE 1: FOUR TYPES OF DIAGRAMS 





in 




:4c 

n 

E 

c 



Flgur# 15-11. In a steam healing 
system liquid water absorbs the heat 
needed to vaporize it. When the vapoi' 
condenses back to a liquid in th^ 
radiator, what happens to this heat 6i 
vaporization? . 

The heat is set free into the room. 



(A) Diagram - Concrete/Oynamlc/Labeled 



CELSIUS 



Bolting 
point 

Freezing 

point 



Absolute 
zero"''* 



.-100^ 



KELVIN 



«^273a^^ 



1 



FAHRENHEIT 
/3\ 



-.373 



-273 



--0 



! 



-212« 



-32* 



—460* 



I 



JHgure 15-6. Temperature is meaS' 
ured on any of three scales: Celsius 
Kelvin, and Fahrenheit. Which two' 
are based on the boiling and (reezinq 
points of water? 

Celsius and Fahrenheil 



(B) Diagram - Concrete/Static/Labeled 
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FIGURE 19-2. 

(C) Diagram - Concrete/Dynamlc/Unlabeled 



nguf€ 16^ Wmt air 

UMnms^ lheskf>mtkks 
mou€ futo' Mnd wUh 
ffwt fott€» The pfts* 
wupt on ifie inMe sot- 
Aide (navMCs Ifie 




(D) Diagram - Abstract/Oynamlc/Unlabeled 
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FIGURE 2: TWO TYPES OF EQUATIONS 



Heat absorbed ^ Change in 
or released 



Specific heat 



, X Mass X 
temperature capacity 

H = At X mx C, 
A means change, so Ar means change in temperature 
A/ equals tz - ti when heat is gained. When heat lost A/ 
equak ti — 



A 



(A) Equations with Words 



(B) Equations with Symbols 



FIGURE 3: TABLE IN LIST FORM 



1^M» 18-1: Specific Hmts of Various Substances 





Spedfk hmt 




Specific (teal 




Su6$tan€€ 




aluminum 


0.92 


Iron 


0.46 • • 


bmss 


OM 


lead 


0.13 


copper 


O.W 


olive oil 


2.0 


class 


OM 


silver 


024 


flycoi 


2j6 


water 


42 


COM 


0J3 


tine 


OM 


ke 


2.1 






l<rc to 


i 







' 4 

U » 
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TABLE 1: FREQUENCY OF SYMBOL SYSTEMS BY KEY CONCEPTS 
WITHIN EACH TEXTBOOK 



CONCEPTS 



Symbo 1 
System 


i 1 


1 2 


3 


4 1 


5 


6 1 7 1 8 1 


V 


1 A 3 
1 B 4 
1 C 7 


1 A 4 

1 C 8 


A 6 
B 2 
C 1 


A 4 1 

B 2 1 
C 2 1 


C 1 


A 6 1 A 
B 6 I B 
C 5 1 


6 1 A 8 1 
5 1 B 9 1 
1 C 8 1 


OADU 


1 






A 1 j 


C 1 


i 


i i 


DCOL 


j 










A 1 1 


1 A 3 1 
1 B 2 1 
1 C 2 1 


• 

ocou 


{ 










\ i 


1 B 1 i 


DCSL 


} 


1 A 2 

1 c 1 








i 1 B 


2 i 1 
1 C 1 1 


TL 


j 




I A 1 
B 1 

1 C 1 






i i 


! i 


EW 


1 






i A 1 








ES 








i A 1 








P 




1 B 2 
1 C 1 




1 A 2 1 
1 B 2 




1 A 4 1 A 
B 1 1 B 

1 C 2 1 


1 1 A 1 1 
1 1 B 2 1 
1 C 1 1 



Textbv 3k8 are labeled A» and C. 

Concepts are: 1. Definitions of heat and temperature; 2. Thermometers and 
temperature scales; 3. Specific heat; 4« Heat transfer; 5. Pressure; 6. 
Molecules and movement; 7. Heating systems; 8. Insulation 

Symbol Systems are: V - Verbal; DADU - Olagram/Abstract/Dynaalc/Unlabeled ; 
DCDL - Olagram/Concrete/Oynamlc/Labeled; DCDU - Dlagram/Concrete/Oynamlc/ 
Unlabeled; DCSL - Dlagram/Concrete/Stat Ic/Labeled ; TL - Table In List Form; 
EW - Equation vrlth Words; ES - Equation with Symbols; P - Photograph 
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3^^LE 2: FREQUENCIES OF SYMBOL SYSTEMS 
SYMBOL SYSTEM 



FOR ALL KEY CONCEPTS 
FREQUENCY 



Verbal 

Diagram - Concrete/Static/Labeled 
Dlagraw - Concrete/Dynamic/Labeled 
Diagram - Concrete/Dynamlc/Unlabeled 
Diagram - Abstract/Dynamic/Unlabeled 
Equation - Words 
Equation - Symbols 
Table - List 
Photograph 



97 paragraphs 
4 diagrams 
4 diaprams 

1 diagram 

2 diagrams 
1 equation 
1 equation 

3 tables 

12 photographs 
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